Video coders are well known devices. They code a sequence of image data (“source video data”) according to predetermined coding processes for presentation of coded video data to a channel. The coded video data often occupies less bandwidth in a channel than the source video data and, therefore, many video coders achieve a high degree of data compression. Many video coding processes are lossy processes; when coded video data is retrieved from a channel decoded, it generates recovered video data that is similar but not identical to the source video data. When source video data is coded, a video coder selects various coding parameters that can affect the quality of the recovered video data at the decoder and the bandwidth occupied by the coded video data in the channel. Additionally, the selection of coding parameters can affect the complexity of coding processes that are to be performed at the video coder and the video decoder.
FIG. 1 illustrates a functional block diagram of a basic video coder 100. A video coder 100 takes data from source video data 110 and codes the source video data 110 to produce coded video data 170 for presentation to a channel 120 to be decoded by a decoder connected to the channel 120.The video coder 100 partially codes the source video data 110 and generates partially coded data 160 and coder parameters 165 for video coder engine 150 to code the partially coded data 160 according to commonly used procedures using the coder parameters 165, e.g. quantizer parameter settings, spatial transforms, run-length coding, and entropy coding.
Typically, the coder 100 generates the partially coded data 160 and selects the coder parameters 165 to balance a set of constraints including: 1) the quality of recovered video, 2) the bit rates available from the channel 120, 3) the complexity available at the video coder 100, and 4) the complexity available at the decoder. The complexity of the decoder needs to be considered because the decoding process is performed on the decoder, and the resources available on the decoder are often limited.
The video coder 100 performs coding processes according to a coding policy to generate the partially coded data 160 and the coding parameters 165. Commonly, the coding processes operate on the source video data 110 on a frame-by-frame basis in which the video coder 100 parses a frame into a plurality of pixel groups (e.g., 8×8 or 16×16 blocks of pixels) and codes the frame on a pixel group by pixel group basis. For each pixel group being coded, the video coder 100 may attempt to predict the pixel group from other pixel groups of the current frame (I or intra coding) or from pixel groups of other frames (B, P or inter coding). The pixel groups used to predict other pixel groups may be called reference pixel groups. For example, the video coder 100 may code a pixel group by referring to a source pixel group with instructions for applying a difference (residual) to the source pixel group to get an approximation of the pixel group. (e.g., if an object such as a ball were in a scene for several seconds then pixel groups of the ball from earlier in the scene may be used to predict pixel groups later in the scene). Applying a difference may be necessary to compensate for such things as lighting changes, camera angle changes, etc. By using prediction, the video coder 100 may exploit spatial or temporal redundancy in the source video data 110 to code pixel groups with fewer pixels. However, using prediction is a lossy process, and when chains of predictions are used the quality of the recovered video on the decoder may suffer due to propagation errors, and transmission errors may corrupt not only a current frame but also all the frames that are predicted from the current frame. One way to reset these types of errors is to send an I frame, or a frame with pixel groups that only refers to other pixel groups in the current frame. The decoder is reset, because the decoder may only refer to frames after the I frame.
Thus, video coders 100 according to a coding policy assign certain frames from the source video data 110 to be reference frames for prediction purposes whereas others may not be so assigned. The video coder 100 according to a coding policy may also select coder parameters 160 for video coder engine 150 to code the partially coded data 160 according to commonly used procedures, e.g. quantizer parameter settings, spatial transforms, run-length coding, and entropy coding. Quantization parameter selections can strongly influence bandwidth consumption and image quality in recovered video data. Generally speaking, higher quantization parameters can achieve higher compression ratios but can also lower quality of recovered image data. Therefore, the selection of quantization parameters (and other coder parameters) can be influenced by performance constraints such as channel bit rate limits and image quality requirements.
The coded video data 170 is transmitted across the channel 120 to a decoder. The coded video data 170 may be merged into a channel bitstream with data of other pixel groups and frames and with coded data representing other artifacts of the image sequence (such as audio). During transmission the coded video data 170 may be corrupted due to transmission errors relating to the channel 120 and/or the decoder.
When an (intra-coded)I frame is used in video coding, it may reset the state of a video decoder. However, the I frames typically are large compared to inter-coded frames and, therefore, they consume much more bandwidth. The inventors of the present application believe that modern coding processes do not require transmission of intra-coded frames that completely reset a video sequence. Instead, it may be useful to perform such reset operations on a pixel group by pixel group basis. In doing so, it may be useful to base such reset operations on a determination of how often pixel groups of frames are referenced by the encoding of other pixel groups in determining the quality of coding to use for pixel groups. Accordingly, there is a need in the art for apparatuses and methods to reduce the need to send I frames to reset the state of a video decoder by identifying pixel groups for enhanced coding and then enhancing the coding of the identified pixel groups.